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In-flight loss of control (LOC) is currently the leading cause of fatal accidents based on 
various commercial aircraft accident statistics. As the Next Generation Air Transportation 
System (NextGen) emerges, new contributing factors leading to LOC are anticipated. The 
NASA Aviation Safety Program (AvSP), along with other aviation agencies and communities 
are actively developing safety products to mitigate the LOC risk. This paper discusses the 
approach used to construct a generic integrated LOC accident framework (LOCAF) model 
based on a detailed review of LOC accidents over the past two decades. The LOCAF model 
is comprised of causal factors from the domain of human factors, aircraft system component 
failures, and atmospheric environment. The multiple interdependent causal factors are 
expressed in an Object-Oriented Bayesian belief network. In addition to predicting the 
likelihood of LOC accident occurrence, the system-level integrated LOCAF model is able to 
evaluate the impact of new safety technology products developed in AvSP. This provides 
valuable information to decision makers in strategizing NASA’s aviation safety technology 
portfolio. The focus of this paper is on the analysis of human causal factors in the model, 
including the contributions from flight crew and maintenance workers. The Human Factors 
Analysis and Classification System (HFACS) taxonomy was used to develop human related 
causal factors. The preliminary results from the baseline LOCAF model are also 
presented. 


Introduction 

A lthough the overall accident rate has decreased, in-flight loss of control (LOC) is the leading cause of fatal 
accidents. The Boeing statistical summary of connnercial jet aircraft accidents^ shows that 1841 lives (about 
37% of total fatalities) were lost in 20 (out of 87) fatal LOC accidents worldwide in the years 2001-2010. Similarly, 
an examination of the National Transportation Safety Board (NTSB) accident database^ reveals that LOC accidents 
represent more than 10% of all US-based commercial airlines accidents and more than 50% of all fatalities over the 
past two decades. Lurthermore, findings in the LOC accident reports suggest the source of causality is centered 
around human errors. 

In addition to the current contributing factors leading to LOC, the Next Generation Air Transportation System 
(NextGen) as envisioned in the 2025 timeframe is expected to bring forth potential future causes, contributing to 
LOC with decreased separation distance and increased National Airspace System (NAS) capacity. The NASA 
Aviation Safety Program (AvSP) conducted a high-level qualitative analysis of future safety risks^, identifying LOC 
as one of the most critical future risks. A better understanding of LOC causes and prevention strategies are necessary 
to help reduce the fatality rate and to keep the accident rate low in the foreseeable NAS. To this end, the AvSP 
Aviation Safety Analysis Team (ASAT) has been asked to develop a system-level integrated LOC model, not only 
to predict the likelihood of future LOC, but, more importantly, to evaluate the impact of new safety technology 
product insertions on LOC. 

This paper discusses the approach used to develop a generic integrated LOC accident framework (LOCAL) 
model from the causal factor analysis of LOC accidents for the time period 1987-2009. This integrated LOC model 
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was constructed as an Influence Diagram (ID), an extension of Bayesian belief network (BBN), using Hugin^ a 
connnercially available software package. The causal factor analysis focuses on human related contributing factors 
including pilots and maintenance workers. The Human Factors Analysis and Classification System (HFACS) 
taxonomy was used for human related causal factors in the BBN-based LOC model. The decision nodes in the 
model were used to represent proposed new safety technology products currently being developed in the AvSP 
Vehicle Systems Safety Technologies (VSST) project, System-Wide Safety and Assurance Technologies (SSAT) 
project, and Atmospheric Environment Safety Technologies (AEST) project. 

I. LOC Accidents Database Review 

In order to investigate current and future LOC accidents, a set of historical data was considered for review and 
categorization. The Aviation Safety Analysis Team (ASAT)^ at NASA periodically obtains a copy of the NTSB 
accident/incident database from Aviation Safety Information Analysis and Sharing (ASIAS) . All accidents 
between 1987 and 2008 involving fixed-wing commercially built aircraft operating under Federal Aviation 
Regulations (FAR) Part 121, Part 135 or Part 91 have been assigned occurrence categories based on the taxonomy 
developed by the Connnercial Aviation Safety Team/International Civil Aviation Organization (CAST/ICAO) 
Common Taxonomy Team (CICTT)^’"^. Out of this database, the LOC-I accidents for FAR Part 121 and Part 135 
were selected. Additional accidents for 2009 were acquired from the NTSB database and categorized using the 
CICTT taxonomy. 

A total of 316 accidents from Part 121 and Part 135 for the years 1987-2009 constitute the initial LOC accident 
database for the current study. The initial list of LOC accidents was refined based on a set of analysis ground rules 
and assumptions, such as restricting consideration to accidents occurring in the U.S with maximum take-off weights 
(MTOW) exceeding 12,500 lbs. As a result, the final accident list contains 54 LOC accident cases with 1429 
fatalities, 154 serious injuries, and 283 minor injuries for 1987-2009 timeline, which covers 47.7% fatalities of all 
Part 121 and 135 accidents within the same timeframe. 

In order to categorize the 54 LOC cases cited above, the authors mainly employed NTSB accident reports. 
However, for the cases where the sole information source is the less detailed NTSB factual report, an additional 
source of information was found in the Aviation Safety Analysis and Functionality Evaluation (ASAFE) program. 
The ASAFE program uses a modified version of the Human Factors Analysis and Classification System (HFACS)^ 
taxonomy and its maintenance extension (HFACS -ME)^, assigning corresponding causal factor codes via the 
designated computer program. The additional information in ASAFE allowed the authors to further classify the LOC 
accidents associated with human errors within the provided timeframe. 



Figure 1. LOC Primary accident cause distribution. 


II. Generalized LOC Accident Types 

As previously mentioned, the database 
review allowed authors to construct a 
generalized LOC Accident Framework 
(LOCAF). Given the information provided 
from the CICTT categorization, NTSB 
accident and factual reports, and ASAFE, 
these 54 LOC accident cases were reviewed 
for the causal and contributing accident 
factors from three main perspectives; 
namely, the aircraft system, human, and 
external environment. The aircraft system 
components include System Component 
(SC) related accidents arising from 
maintenance, design and vehicle system 
issues. The HUman (HU) component 
includes flight crew and ground personnel. 
The external ENVironment (ENV) relates 
to atmospheric disturbances, such as icing, 
turbulence, etc. All the accidents contain 
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either a single or a combination of these primary components, and were grouped into five main accident categories. 

The number of occurrences of these five main accident categories is given in Fig. 1. Out of the 54 accidents, only 
12 accidents are directly related to an aircraft system or component issue (Type I - SC, 22.2%). The remaining 42 
accidents (78.2%) include human error, either directly or indirectly. In 11 of the cases, both the flight crew and the 
aircraft systems were found to be contributors to the accident (Type II - SC+HU, 20.4%). The human error (both 
flight crew or ground personnel induced) was observed in 14 accidents (Type III - HU, 25.9%). Out of the human 
related accidents, 3 (5.5%) were related to ground personnel, and 11 (20.4%) were associated with flight crew 
deficiencies. Out of all cases, 17 accidents were related to a combination of weather related disturbances, coupled 
with flight crew errors (Type IV - HU+ENV, 31.5%). Finally, four accidents involved all three components, the 
human, environment, and system component issues (Type V - HU+ENV+SCF, 7.4%). 

Although not shown, inappropriate Air Traffic Management (ATM) guidance was associated with 5 accidents, 
cited as findings by the NTSB reports, but not determined as primary factor or cause. Four of the ATM related 
accidents were encountered in environment category (ground/inflight icing, wake turbulence, and conducive weather 
related) and in one accident, ATM guidance contributed to flight crew’s task fixation. 

The next step in the research was to develop a generalized LOG accident framework (LOCAF) embarking from 
the classification given above, which is able to demonstrate the individual accident cases covered in the dataset 
while identifying the interaction among the three major components, human, aircraft systems, and the environment. 

III. Generalized LOG Accident Framework (LOCAF) 

The generalized LOCAF was mainly based on an extensive review of the historical accidents given in the 
database review section. LOCAF provides the foundation for the LOC model. Fig. 2 illustratres the accident 
framework as a generalized interaction of various factors given in the boxes, leading to a LOC accident. 



Figure 2. LOC Accident Framework (LOCAF). 

The database review indicated that the aircraft system components, environmental conditions as well as ATM 
operations can have a negative impact on the flight crew performance. For instance, out-of-routine aircraft system 
indications, changing environmental conditions, or deficient ATM guidance can distract the flight crew, especially 
during the critical phases of flight. Such distractions, in turn, can lead to performance overload, confusion, panic, 
task fixation and/or saturation, captured in the Task Fixation box. However, even without the presence of such 
external disturbances, the flight crew can experience spatial disorientation or suffer from fatigue or medical illness. 
The Preconditions for Unsafe Acts box depicts initial flight crew deficiencies (a separate framework will be given in 
Section A, providing a more detailed review). 

Once the initial flight crew performance is updated via the presence of external disturbances, the information is 
incorporated into to the Upset box with two other accident contributors; the system component failures and 
environmental factors, are taken into consideration. Environmental Conditions depicts accidents where convective 
weather, wake turbulence, icing, and bird strikes are determined as factor/cause. Similarly, the System Component 
Issues box contain failures or malfunctions that are a primary cause/factor in the LOC event. The Upset box depicts 
that the aircraft is an abnormal flight condition, outside the normal flight envelope. The cause for the upset can be 
stemmed from the the flight crew alone (e.g. loss of speed due to deficient instrumentation scan), system component 
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failure (e.g. engine, hydraulic or software failure), environmental factor, or a combination of these factors. The five 
accident types identified in the previous section are represented in the Upset box. Finally, the flight crew’s failure to 
correct the upset using conventional techniques will eventually lead to a LOC accident. 

A. Flight Crew Performance Deficiency Encapsulation using HFACS 

Historically, the Flight Crew (FC) is the most important contributor in aviation accidents^. The predispositions of 
the flight crew (e.g. issues with experience/training, physical or mental conditions, etc.) was given in the 
Preconditions for Unsafe Acts box in Fig. 2. The underlying framework for FC predispositions is captured via a 
modified HFACS framework^. The hierarchical causal relationship structure within HFACS allows flight crew 
errors to be traced all the way to the organizational level. The HFACS architecture consists of 3 levels; Operator 
Organizational Influences level. Operator Unsafe Supervision level, and finally the deficiency at the Flight Crew 
level. 

In order to construct the FC performance deficiency framework, 39 flight crew related accidents (out of the 
original 54 cases) were analyzed and classified based on the causal factor definitions given in the HFACS 
taxonomy. Figure 3 provides the original HFACS causal factors and the occurrence numbers and frequencies of each 
factor. Despite the small sample size, the correlation between causal factor pairs is used to determine the link 
connecting the causal factors to each other^^ 






r N 

Operator (Airline) 


Operator (Airline) 


Flight Crew 

Organizational Influences 


Unsafe Supervision 


Deficiency 





^ ^ 



Planned Inapproriate Operations 
(Manning - Crew Scheduling) 
[5 cases, 13% of all FC] 


Adverse Physiological States 
(Fatigue/Illness/Disorientation) 
[8 cases, 21% of all FC] 


Figure 3. Flight Crew Performance Deficiency Framework based on HFACS 

The organizational influences are divided in three groups; Organizational Climate, Organizational Processes, 
and Resource Management. The organizational climate refers to the working atmosphere within an organization as 
well as values and beliefs. It includes the organizational structure (chain-of-command, delegation of authority, 
connnunication practices), hiring-firing and promotion policies, and culture (norms and rules, etc.). It is argued that 
the individual errors within the organization can be closely related to the organizational climate^. Organizational 
climate broadly affects the organizational processes (such as the operational tempo, time pressure, production 
quotas, standards and procedures, documentation, risk management, safety programs oversight, etc.). Similarly, 


The total number of flight crew accidents (39 cases) constitutes the sample size for the correlation analysis. 
Correlation Analysis within Excel’s Data Analysis ToolPak Add-in was performed for all the accidents and results 
were displayed on the links in Fig. 3. 
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resource acquisition management (staffing/manning, training, cost cutting, funding issues, and equipment and 
facility issues) is closely related to the safety understanding and safety policy which is a part of the organizational 
climate^. 

The organizational structure influences the supervision performance at the mid-level management. Although 
HFACS indicates four components within this level, the investigation did not identify supervision violations within 
the database. The remaining three components; Failed to Correct a Known Problem, Inadequate Supervision, and 
Planned Inappropriate Operations constitute the underlying causes that yield to preconditions for unsafe acts. Issues 
encountered with lack of oversight, training, guidance or performance tracking are associated with inadequate 
supervision. Inappropriate operations include cases where the crew was not allowed to rest adequately due to 
improper scheduling or improper manning took place. Errors in documentation, identification of risky aviators, or 
reporting unsafe practices are covered in Failed to Correct a Known Problem node. The influences of unsafe 
supervision are observed within the cases that make up the preconditions of unsafe acts. These conditions are the 
cases where the members of the flight crew were not good candidates (due to physical & mental limitations) or 
possessed adverse mental states or pernicious attributes (over or under confidence in abilities and skills). Similarly, 
issues encountered with lack of training/experience as well as physiological states such as fatigue, medical illness, 
spatial disorientation, and crew resource management can be attributed to the combination of unsafe supervision 
deficiencies^, as shown in Fig. 3. 

B. System Component Issues 

The review of the SC related accidents (27 cases) revealed that faulty design was the primary cause of eight 
accidents (30%) whereas deficiencies within the maintenance processes caused ten accidents (37%). The remainder 
nine accident cases (33%) include failures associated with avionics, instrumentation, mechanical, and other systems 
as well as undetermined failures that are not identified as either maintenance or design related (Fig. 4). Within the 
maintenance related accidents, four cases were associated with powerplant (control linkages, propellers, engine 
rotors, etc.), two structures (airframe, wings, stabilizers, etc.), and four flight controls (flaps, ailerons, spoilers, 
elevators, etc.). Maintenance related accidents were often related to the maintainer’s failure to notice fatigues, 

cracks, corrosion pits, etc. (3 cases or 
11% of all SC) or due to ineffective 
maintenance (such as installing an 
incorrect part, misadjustment of 
components, incorrect rigging of 
flight control surfaces, etc.) (7 cases 
or 26% of all SC). 

Similarly to the maintenance 
related system component failures, 
design related accidents were also 
divided in subcategories based on the 
presence of an actual component 
failure. In three cases (11% of all SC), 
the system component in discussion 
worked as planned, but the operation 
caused a hazardous condition for the 
aircraft. An example for this category 
can be an inappropriate software logic 
design in the windshear warning 
system that desensitized during flap 
operations and failed to alert the flight 
crew. The other type of design failure 
is the actual failure of the component 
due to its design (5 cases, 18% of all 
SC). 

The final section of SC related 
accidents is the SCF-X, including 
failures/malfunctions that couldn’t be 



Figure 4. System Component Related Accidents 
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associated with either a design or maintenance related root-cause. This subcategory is divided between hardware/ 
mechanical failures, avionics/ instrumentation failures, and undetermined reasons. In many cases, accident 
investigations does not provide information regarding the underlying cause of the failure. 


^ Takeoff Icing j 


^ In-Flight Icin^ 


C. Environmental Conditions 

Environmental disturbances including icing, thunderstorms, turbulence and wake turbulence were either a cause 
or a factor in 17 accidents (31%) of all LOC accidents (Fig. 5). 

It is important to note that within the database, all weather 
related accidents had also a flight crew error as a co-factor. 

The probable reason for the lack of any unrecoverable weather 
related accidents can be due to the omission of aircraft with 
MTOW less than 12,500 lbs from the database. Out of all 
weather related accidents, 76% include icing related problems, 
which occur during takeoff or in-flight. Loss of control during 
takeoff due to icing is caused by the presence of ice on the 
airframe and wings before takeoff. During the takeoff roll, the 
presence of ice decreases lift due to disrupted airflow around 
the airfoil, causing the aircraft to yaw sharply or fail to gain 
speed following the rotation (4 cases or 23%). 

Loss of control due to in-flight icing usually occurs on the 
leading-edge of the airfoil and empennage, increasing drag 
and decreasing lift. Degraded aircraft performance and speed 
eventually yield to a stall (9 cases or 53%). Accidents related 

to adverse weather/wind, turbulence, and windshear present in Figure 5. Environment Related Accidents 
three cases (18%). Finally, one accident was caused by wake 

turbulence encounter, where a B757 aircraft was followed by a smaller lAI Westwind aircraft. 



IV. Literature Review on Safety Methods and Tools 

The generalized accident framework within LOCAL was used as a platform to model LOC accidents. In order to 
select the modeling approach, a literature review of current safety methods and tools was conducted. A safety 
methods database maintained by National Aerospace Laboratory Air Transport Safety Institute^ (NLR-ATSI) in the 
Netherlands contains over 700 methods. These methods^ and software tools are developed for various domains of 
application. Review of this database revealed that more than 100 methods and tools are associated with aviation 
applications, to discover and prevent primary accident causes and prevent them in the future. However, rather than 
an exhaustive review of all the identified methods, only certain popular methods and their key components were 
investigated and compared. 

Pertaining to aviation safety risk assessment, the prominent methods include the Aviation Safety Risk Model 
(ASRM)^®, the Causal model for Air Transport Safety (CATS)^\ and general Probabilistic Risk Assessment (PRA 
based on mainly event tree and fault tree analysis). The development of ASRM was supported by the FA A and the 
NASA’s Aviation Safety Program (AvSP), whereas CATS was supported by the Netherlands Ministry of Transport. 
CATS employs a combination of three modeling techniques. Event Sequence Diagrams (ESDs), Fault Trees (FTs) 
and Bayesian Belief Networks (BBN), whereas ASRM uses only BBNs. The ESDs and the FTs in CATS, though 
developed separately, are converted into BBNs, and the final risk model is mathematically represented as a single 
integrated BBN. CATS employed UNINET software to program BBNs, while Hugin was the software choice for 
ASRM. 

For this study, the key objective is to develop a system-level integrated LOC model to examine the integration of 
AvSP portfolio elements to address the future risks of LOC. The LOC accidents rarely result from a single linear 
causal sequence due to interactions between aircraft system, humans and external environment variables. Therefore, 
a BBN is a more intuitive and appropriate method for this large complex integrated LOC model than traditional FT 
and ET methods that do not easily capture the multiple non-linear dependencies. In the BBN framework, 
probabilistic and causal relationships among variables are flexibly represented and executed as graphs, and can thus 
be visualized and easily modified. This facilitates model building, and rapid interactive manipulations of the model 
to explore the causal features, which are particularly helpful when interacting with Subject Matter Experts (SMEs) 
in the probability elicitation process. 
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There are many different BBN software packages^^ available from both commercial vendors and the public- 
domain, such as BNet, Hugin, Netica, WinBUGS. Hugin software has many advantages over competing tools for 
the purpose of this modeling endeavor^^. Hugin is favored mainly due to three reasons: its critical capability of 
employing Influence Diagrams (ID), the Object-Oriented Bayesian Network (OOBN) feature, and the availability of 
an Application Program Interface (API). ID is an effective means to evaluate the impact of new safety technologies 
on the reduction of aviation safety system risk, providing decision makers useful information for technology 
portfolio management. The OOBN feature allows construction of a modular and hierarchical BBN for more efficient 
modeling, and succinct presentation of a large and complex BBN. The Hugin API is available for most major 
programming languages, such as C, C-h-h, Java, etc. so that additional modules can be developed to enhance the core 
BBN model component. 

V. BBN-based Generic LOG Model with Human Error Emphasis 

A previous NASA-guided study^® with a similar goal of evaluating the effectiveness of NASA safety technology 
products employed case studies to represent various types of accidents (e.g. LOG, controlled flight into terrain, 
runway incursion, engine failure, etc.). However, the prominence of human factors, especially the human error 
interactions in all types of LOG accidents, exceeds the ability of a case study approach to be used here. 

The Hugin BBN-based generic LOG model allows the authors to capture conditional probabilities (elicited from 
SMEs) to compute the likelihood of LOG. More importantly, the generic LOG model provides a platform to evaluate 
the effectiveness of currently developed NASA safety technology products within the VS ST, AEST, and SSAT 
projects while allowing enough flexibility to the authors to modify the generic human error model to address future 
concerns without being restricted to any specific case study. Eor that reason, the previously defined LOGAE 
framework (Eig. 2) was converted into a Bayesian based Object Oriented model using Hugin Software. The use of 
OOBN simplifies construction and maintenance of this large and complex LOGAE model. 

Eig. 6 provides the causal factors of the accident, represented via nodes and connected by links, designating 
dependencies at the top-level of the LOGAE model. The ellipse nodes are called discrete chance nodes and the each 
node’s dependency on its parent nodes are given with a Gonditional Probability Table (GPT). GPTs include all the 
possible combinations of nodes’ parents and the respective probability values used in Bayesian calculations. 

The rectangle nodes with rounded corners in Pig. 6. represent instance nodes, indicating they are encapsulated. 
The encapsulated nodes include Flight Crew Conditions, Environmental, and System Component Failures and each 
of these nodes contains their own separate BBN (also called subnets) which are fed to this level. The application of 
object-oriented encapsulation allows each subnet to be enhanced or replaced easily, as well as the addition of new 



Figure 6. Top-level LOCAF Model Representation using Hugin 
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sub-models, without disturbing the rest of network in the future. Within the scope of this paper, only the flight crew 
conditions and system component failures associated with faulty maintenance (SCF-M) will be provided in detail 
within Sections V-A and V-B. The model shown in Fig. 6 coupled with the encapsulated nodes constitute the 
baseline LOC model. The details and the preliminary results of the baseline model will be discussed in Section V-C. 
The rectangle nodes are called decision nodes and they are used to represent various NASA products and 
technologies (X-1, Y-2, X-3, etc.). It should be noted that the product label and the placement of the decision nodes 
in Fig. 6 are notional. Nevertheless, the process of product insertion and evaluation will be covered in Section V-D. 

The encapsulated Flight Crew Conditions node provides the calculated probability of deficiency in flight crew’s 
initial performance. The presence of deficiency indicates the probability of flight crew errors (skill, decision, and 
perception) and violations in an accident. This node depicts the flight crew’s preexisting conditions before the flight 
and is covered in more detail in Flight Crew Performance Deficiency section. 

The Task Fixation/Saturation is an aggregating node, combining the possible effects of external influences on 
the flight crew. It covers the presence of environmental factors (e.g. presence of low-ceiling, wind gusts, or icing 
conditions), discrepancies with the technological environment (interactions with the a/c, autopilot/automation, 
cockpit/checklist design, etc.), and the ATC related issues (increased workload caused by inadequate ATC 
procedures/connnunication or due to high traffic density). The resultant effect is calculated within the task 
fixation/saturation node which has three states; fixation, saturation, no effect. Environment, technological 
environment and ATC issue nodes are two-state nodes where the effects of each node on an accident are obtained 
from SMEs. 

The Updated Flight Crew Performance Deficiency node provides the flight crew performance deficiencies 
during the flight by taking various effects into consideration. The link from the Task Fixation/Saturation node is fed 
into the Updated Flight Crew Performance Deficiency in order to calculate the ongoing flight crew performance 
deficiency which may be impaired from the effects of fixation/saturation. In other words, any presence of task 
saturation/fixation effect is aggregated when calculating the updated flight crew performance deficiency. It was 
often observed that the presence of external effects such as low visibility or increased traffic workload were not 
directly related to accidents but may have contributed to flight crew’s performance degradation. 

The Upset/Abnormal Flight Conditions node is another aggregating node where inputs from System Component 
Failures (SCF), Environmental factors, and FC performance subnets are evaluated to provide the probability of an 
upset/abnormal flight condition occurrence. An upset condition includes engine failure, fire, stall, wake, 
uncontrolled altitude deviation, unresponsive aircraft, etc. where one or more parent nodes can be a factor. For 
example, a deficiency in flight crew’s skill can result in a loss of aircraft energy (low-speed) which can cause a stall. 
Similarly, a SCF within the power-plant or aircraft controls can render the aircraft uncontrollable, which in turn 
yield to an abnormal flight condition. The encapsulated environment node includes the presence of wake 
turbulence, convective activity, icing and wildlife/bird strike. Similarly SCF provides failure probabilities of a 
generic aircraft that contributes in an accident scenario. 

The Recovery Failure node (successful/unsuccessful recovery) refers to the ability of the flight crew to recovery 
from an abnormal flight condition. Failure to recovery inherently leads to a LOC accident, where a successful 
recovery indicates the absence of LOC. Inherently, deficiencies in the flight crew’s performance are directly related 
to the recovery failure (i.e. following an emergency procedure, flying skills, training, etc.). 

A. Flight Crew Performance Deficiency Subnet 

The flight crew performance deficiency framework which was based on a modified version of the HFACS^’^ 
architecture was given in Fig 3. The contents of the initial framework was used as a baseline for the development of 
flight crew performance deficiency subnet as shown in Fig. 7. This subnet was then evaluated and modified by the 
SME panel^^. This subnet provides the initial flight crew deficiency probability to the top-level LOCAF given in 
Fig. 6 using Hugin’s output node function. 

The Flight Crew Performance Deficiency subnet is divided into three organizational hierarchy levels (from left- 
to-right); airline organizational influences (Organizational Climate, Organizational Process, Resource Management), 
mid-level management/supervision (Failed to Correct a Known Problem, Training, Inadequate Supervision, 
Inappropriate Operations), and finally the flight crew’s conditions (Personal Readiness and CRM). 
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Figure 7. Flight Crew Performance Deficiency Subnet 

1. Airline Organizational Influences 

The Organizational Climate node designates the working atmosphere within the airline company. It refers to the 
efficiency with which the processes related to organizational communication, chain-of-command, and delegation of 
authority are performed. Also the organization’s safety culture (unspoken rules, norms and values, attitude towards 
safety) and policies (official guidelines such as promotion, retention, sick leave, etc.) are covered under this node. 
This node contains two-states; probability of Organizational Climate Issues, v, and the absence of organizational 
climate issues, 1-x. The values for the two states are obtained from the SMEs. 

The Organizational climate sets the tone for both organizational processes and resource management in the 
airline company. Organizational Processes involve corporate decisions and rules that govern everyday activities 
within an airline company including operations, oversight, and procedures, (e.g., organizational tempo, incentives, 
time pressures, and schedules, safety/risk program establishment performance standards, documentations, and 
instructions. The two-state nature of this node points out deficiencies on airline’s organizational processes (present 
or absent). 

The Resource Management node refers to the corporate-level decision making regarding the allocation and 
maintenance of organizational assets such as human resources (e.g. personnel selection, staffing/manning, training), 
monetary assets, facilities, and equipment. Examples of deficiencies associated with underperforming resource 
management can be given as lack of funding, excessive cost cutting, opting for low-cost but sub-optimal 
alternatives, inexperienced flight/maintenance abundance, etc. 

2. Supervision Level Influences 

Based on the review of the accidents, it was observed that deficiencies at high-level organizational management 
propagated to the supervision level. At the supervision level, the Failed to Correct a Known Problem node refers to 
the instances when known deficiencies among individuals, equipment, operations, procedures or other safety areas 
are left untreated. Supervision’s failure to identify and correct the risky behaviors, safety hazards and failure to 
initiate and follow up corrective action is categorized under this node. 

The Training node covers deficiencies associated with a supervisors’ failure to provide adequate training due to 
cost cutting, scheduling conflicts, or other reasons. Also included in this node are instructional errors or omissions 
during training as well as inappropriately planned simulator and flight training causing the pilot not to receive 
appropriate knowledge and skill set necessary to safely fly the aircraft. 

The Inadequate Supervision node reflects supervisory deficiencies that take place within organizations. 
Supervisors’ duties include providing guidance, leadership, oversight, and incentives in order to ensure the efficient 
and safe working environment. Examples of inadequate supervision can be cited as failure to provide an adequate 
rest period, lack of accountability, loss of supervisory situational awareness, and failure to track qualification and 
performance of employees. 

The Inappropriate Operations node considers deficiencies associated with the operational tempo and/or 
scheduling of the aircrew. Inappropriately planned operations put the flight crew (and the passengers) at an 
unacceptable risk since crew rest is jeopardized and ultimately performance is adversely affected. This node also 
includes poor crew pairing (seniority issues), excessive tasking/workload, failure to provide adequate brief time, etc. 


9 

American Institute of Aeronautics and Astronautics 


3. Flight Crew ’s Conditions 

Although historical data provided statistical correlation on the presence of various flight crews’ preconditions for 
unsafe acts, the SMEs re-organized this section of HE ACS to contain two parts; Personal Readiness (covering all 
the physical and mental preconditions) and related deficiencies. 

The Personal Readiness node includes all long and short term physical and mental readiness of the flight crew as 
well as the experience and training deficiencies that may be present. Examples of mental limitations and states 
include complacency, distraction, get-home-it-is, misplaced motivation, mental tiredness, distraction, confusion, 
depression, alcoholism, over/lack of confidence. Similarly, examples of physical states and limitations include 
physical fatigue (lack of sleep or flying schedule), medical illness, excessive physical training, impaired 
physiological state, physiological incapacitation, self medication, violation of crew rest requirement or bottle-to- 
throttle requirement. Einally, individuals that are not compatible with aviation (physically or mentally unable to 
process, react of fit to fly due to insufficient reaction time, visual limitation) are also covered in this node. 

The second precondition for unsafe act besides the Personal Readiness is Crew Resource Management (CRM) 
deficiencies. The CRM node includes communication skills and coordination that take place within the flight crew as 
well as between other entities (i.e. other aircraft, ATC, maintenance facility and other support personnel) before, 
during, and after the flight. Regardless of the experience of the pilots, poor CRM and coordination can lead to 
confusion and poor decision making in emergency or trouble shooting situations. 

The resultant deficiencies within the pilot’ s personal readiness and CRM issues determine the value of the Flight 
Crew Performance Deficiency node. This node is the output node of the Flight Crew Performance Deficiency subnet 
and it takes the Bayesian probability values from the parent nodes to calculate the preconditions of the flight crew. 
A deficiency in the flight crew performance is defined as flight crew’s tendency to make errors and commit 
violations. The flight crew errors can be associated pilot’s skills, decision making abilities as well perceptions (e.g. 
stick-and-rudder capabilities, deficient instrument scan, takeoff/landing decision, improper flight planning, 
misjudging distance, descent, altitude, spatial disorientation, vertigo, etc.). 

Unlike errors, violations are performed willingly by bending the rules habitually or in isolation. The Flight Crew 
Performance Deficiencies node captures all the error and violation types covered above and indicates the flight 
crew’s tendancy to perform unsafe acts . As previously mentioned, this provides the initial conditions of the pilots at 
the top-level LOCAE model and it is “updated” with the Task Fixation/Saturation node. 

B. Maintenance Related System Component Failure (SCF-M) Subnet 

Within the scope of this paper, accidents caused by faulty maintenance are taken into consideration due to the 
human factors involved. The SCE-M subnet is a simplified and modified version of the HE ACS -Maintenance 
Extension (HEACS-ME)^. Similar to the HE ACS framework, the maintenance extension considers latent conditions 
such as supervisory deficiencies, maintainer’s preconditions (such as readiness, fatigue/illness, etc.) as well as the 
working environment when analyzing the maintenance related system component failures. Like the flight crew 
preconditions, any deficiency present in the maintainer’s conditions can cascade as an error in one of the four 
generic tasks (adjustment, installation, service of aircraft, and inspection). This subnet covers the repair stations as 
well as the corporate airlines maintenance departments (Eig. 8). 
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Figure 8. System Component Failure related to Maintenance (SCF-M) Subnet 

In order to capture system failures caused by faulty maintenance, this subnet takes into account deficiencies 
within the original equipment manufacturers (OEM), FAA, and airline operators and/or repair stations. The 
following paragraphs provide an overview of the nodes involved in the hierarchy. 

The FAA Certificate Management and Oversight node covers the efficiency with which FAA oversight is 
applied when enforcing rules and regulations such as the Parts 145 (Repair Stations), 21 (Certification Procedures 
for products and parts), 25 (Airworthiness Standards - transport category) as well as other maintenance related 
processes. The FAA’s oversight mandates the compliance of maintenance departments, repair stations as well as 
OEMs to these regulations. Eike the rest of the nodes in this subnet, the two- states designates potential deficiencies 
in the nodes’ function (Yes/No). 

The Manufacturer Management Conditions node refers to the management conditions at the aircraft 
manufacturers, vendors and part suppliers. This node captures the deficiencies within the manufacturer management 
conditions which may lead to inadequate information flow to the end users (airline operators, repair stations) 
regarding the maintenance practices (e.g. late issued service letters/advisory circular/service bulletins, unclear 
maintenance instructions, etc.) Consequently, poorly connnunicated maintenance requirements can result in 
inappropriate maintenance procedures to be performed. 

The Company /Operator Management Conditions node captures the entire management deficiencies that can be 
encountered in maintenance departments and repair stations. Eike the manufacturing companies, maintenance 
departments are also mandated by FAA regulations. Inappropriate FAA oversight (conducted via Repair Station 
Certificate) can cause deficiencies within the repair facility to go unnoticed. Similarly, any problems in guidance 
and connnunication of the OEM can play a significant role in maintenance related system component failures (e.g. 
relaying information regarding changes to the maintenance processes, materials, specifications, etc). For that reason, 
the Company /Operator Management Conditions are affected by both FAA Certificate Management and Oversight 
and by deficiencies stemmed from Manufacturer Management Conditions. Issues within the Company/Operator 
Management Conditions can be given as inadequate documentation of requirements/ processes, resources (e.g. lack 
of job specific tools and equipment, maintenance crew qualifications and training) as well as inappropriate 
implementation of the regulations (e.g. omitting steps in procedures) and supervisory issues. 

The Maintainer Conditions node covers all the physiological and mental condition (readiness, training, 
experience) of the maintainer (e.g., mechanic, electrician) as well as the working conditions (environment, 
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equipment, and workspace). The Company/Operator Management conditions directly affect the aspects of the 
maintainers’ conditions via supplying adequate tools, guidance, and documentation as well as providing sufficient 
time and training for the job. Consequently, deficiencies encountered at the airline’s maintenance department or 
repair facility can be related to the technicians less-than-adequate performance. Issues in the technicians’ 
performance include potential errors (due to lack of attention, knowledge, skill or decision making) and violations 
(routine or exceptional). This node is synonymous to the Personal Readiness node found in the flight crew 
performance subnet. 

The Adjustment/Installation/Service of Aircraft/Inspection (Diagnostics) nodes include the common maintenance 
practices routinely performed on the aircraft. Their goal is to bring the specifications of the part/subsystem to the 
OEM requirements. The aircraft often goes through a non-destructive inspection where the deficiencies are detected 
and the subsystems/parts are installed and/or adjusted. Also, the aircraft undergoes several scheduled (line checks) 
and unscheduled (pilot initiated maintenance write-ups) checks, which are captured within the service of aircraft 
node. The deficiencies found at the maintainer level can have adverse effects on how these four maintenance 
processes are performed and the varying probability values are obtained via SME sessions. 

C. Baseline Model Results 

As previously mentioned, the SME sessions were conducted during the construction of the LOCAE model^^, 
reviewing the nodes and the linkeages, as well as collecting the probability data for the CPTs. Subsequent to the 
conversion of LOCAE to the BBN and data collection via SME sessions, the resulting model without product 
insertion provides the representative baseline for LOC accidents. The preliminary results of the model showed that 
individual subnets, as well as the top-level LOC model was within approximately 3% of the historical data with 
which the model was constructed. Eollowing the calibration/adjustment of the raw data supplied by the experts, the 
LOCAE model was compared with the results obtained from two datasets^^ The overall LOC probability was found 
to be 15.9% (versus historical 13.8%), and 10% (versus historical 12.8%) in the two datasets . The object-oriented 
structure of the model allows authors to test various scenarios where individual subsystems coupled with various 
environmental factors impact on the LOC occurrence. This detailed analysis of the collected data will be further 
analyzed in the later stages of the research. 

D. The Assessment of AvSP Products on LOC 

In addition to computing the likelihood of LOC, the LOCAE model was developed to evaluate the overall 
effectiveness of over forty NASA developed technologies and methodologies (referred to as products) within the 
AvSP. The generic product insertion is a three-step process. Eirst, similarly to the model construction and data 
collection phases, the NASA products was reviewed by the same set of SMEs using product descriptions and 
other documents provided by AvSP Projects. Once the experts obtained a common knowledge over the NASA 
products developed under VSST, AEST, and SSAT Projects, the next phase involved the placement of these 
products into the LOCAL model. The product insertion involves linking the decision node(s) (rectangular nodes, as 
shown in Eig. 6) to causal factors (chance nodes) within the model. The decision nodes have two states; designating 
the presence or absence of the product. The third step of the product insertion process involved acquiring the new 
deficiency probabilities of the nodes considering the presence of the product. The products are assumed to be 100% 
effective, as intended by their designers. The effect of the product on the causal factor is captured within the 
respective node’s CPT. The future phases of this study will include the assessment of various combinations of AvSP 
portfolio elements to determine their effectiveness by calculating the relative change on LOC. 

VI. Conclusions 

Based on historical data, LOC accidents play a significant role in fatal commercial aviation accidents. This paper 
encapsulates the comprehensive process of a generic LOC model building effort which enabled the evaluation of the 
impacts of various NASA AvSP products. The process of model building was initiated with the extensive review of 
a limited, but significant dataset of accidents from 1987 to 2009. The inferences and knowledge acquired from the 
historical data review were used to construct a generic LOC accident framework, which was supported by HLACS 
and HLACS-ME frameworks aiming to capture the human errors encountered with flight crew and faulty 


The databases used for comparison are the 54 accident cases employed to construct the model as well the dataset 
covered in Ref. 3 (Part 121/135, 1988-2004). 

The three experts who participated in the model development and data collection session were also invited to 
insert and evaluate NASA developed AvSP products for consistency purposes. 
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maintenance, respectively. The analysis of the historical data showed that the deficiencies at the airlines’ 
organizational levels are often the underlying cause of flight and maintenance crew related accidents. Consequently, 
the authors developed a high-level airline organizational hierarchy to trace and identify the deficiency propagation. 

The causal accident factors and their interdependencies were constructed using historical data as well as 
aforementioned human factors framework suites. The resultant accident framework was then converted into a 
OOBN using the Hugin Software, which enabled the authors to develop different network fragments of the model 
(human error, system component failure, etc.) separately, then merging to form a generic LOC model. The 
application of OOBN on the LOCAF model allows authors to export entire subnets (e.g. flight crew performance 
subnet) to other OOBN accident models for a broader accident/incident modeling portfolio. The preliminary results 
indicate that the S ME- supported LOCAF model with human factors emphasis is adequate for representing LOC 
accidents, as well as providing a platform for gauging various NASA technologies’ effects on decreasing future 
LOC occurrences. 


Acknowledgment 

The authors would like to thank the entire Aviation Safety Analysis Team, specifically. Dr. Sharon M. Jones, 
Mary Reveley, Lawrence Green (NASA), Joni Evans (AMA, Inc.), and Dr. James Luxhpj (LCR) for their 
contributions to the development of LOCAF model. We are also very grateful to the subject matter experts for their 
participation in the knowledge elicitation sessions and insightful discussions on the model. 

References 


^Boeing Commercial Airplanes, "Statistical Summary of Commercial Jet Airplane Accidents, Worldwide Operations, 1959- 
2010," June 2011. URL: http://www.boeing.com/news/techissues/pdf/statsum.pdf . [Accessed December 2011]. 

^Jones, S. M., Reveley, M. S., Withrow, C., Evans, J. K., Barr, L. C., and Leone, K., "System Analysis of NASA Aviation 
Safety Program - Final Report," (Draft), Hampton, VA, June, 2010. 

^Evans, J. K., "An Application of CICTT Accident Categories to Aviation Accidents in 1988-2004," NASA CR-2007- 
214888, 2007. 

"^CAST/ICAO Common Taxonomy Team, "Aviation Occurence Categories Definitions and Usage Notes," Version 4.1.5, 
April 2011. 

^Shappell, S. T., and Weigmann, D. A., "The Human Factors Analysis and Classification System," U.S. Department of 
Transportation Office of Aviation Medicine, DOT/FAA/ AM-00/7, Washington, DC, February, 2000. 

^Schmidt, J. K., Lawson, D., and Figlock, R., "Human Factors Analysis & Classification System - Maintenance Extension 
(HFACS-ME) Review of Select NTSB Maintenance Mishaps: An Update," 2003. URL: 

https://www2.hf.faa.gov/dtsearch/dtSearch.html . 

^Wiegmann, D. A. and Shappell, S. A., A Human Error Approach to Aviation Accident Analysis: The Human Factors 
Analysis and Classification System. Surrey, UK: Ashgate, 2003 

^NLR, "Safety Methods Database," December 2010. URL: http://www.nlr.nl/documents/flyers/SATdb.pdf . 

^Everdij, M. H. C., Blom, H. A. P., and Kirwan, B., "Development of A Structured Database of Safety Methods," 
Proceedings of the 8th International Conference on Probabalistic Safety Assessment & Management (PSAM8), New Orleans, 
LA, May 14-18, 2006. 

^^Luxhoj, J. T., Jalil, M., and Jones, S. M., "A Risk-Based Decision Support Tool for Evaluating Aviation Technology 
Integration in the National Airspace System," Proceedings of the AIAA's 3rd Annual Aviation Technology, Integration, and 
Operations (ATIO) Technical Forum, Denver, CO, November 17-19, 2003. 

^^NLR Air Transport Safety Institute, "Causal Model for Air Transport Safety, Final Report," Amsterdam, NL, 2009. 

^^Shih, A. T., Ancel, E., Jones, S. M., “Object-Oriented Bayesian Networks (OOBN) for Aviation Accident Modeling and 
Technology Portfolio Impact Assessment” Proceedings of the American Society for Engineering Management (ASEM) 33^^ 
International Annual Conference, Virginia Beach, VA, October 17-20, 2012 (submitted for publication). 

^^Luxhoj, J. T., Shih, A. T., Jones, S. M., Ancel, E. and Reveley, M. S., "Safety Risk Knowledge Elicitation in Support of 
Aeronautical R&D Portfolio Management: A Case Study," Proceedings of the American Society for Engineering Management 
(ASEM) 33"^^ International Annual Conference, Virginia Beach, VA, October 17-20, 2012 (submitted for publication). 


13 

American Institute of Aeronautics and Astronautics 



